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FOREWORD t> • 

This, booklet has been prepared as part of a United States Education 
•Department (ED) sponsored project on evaluation in early childhood Title 
I (ECT-I) programs .(^t is ohe of a series of resource books developed to 
meet the need expressed by\tate and local personnel for information to- 
, .help them evaluate and improve their early childhood Title I programs. 

> 

The series describes the potential usefulness of various options in making 
, local decisions about program practice. It focuses on the Allowing * > 
questions: • • 

• IVho willj^use the evaluation results? / 

• 'What kinds of decision will the evaluation res/ilts be used for? 

• Does the importance of such use justify the cost of evaluation? 
Together, the resource books address a range of issues relevant to the 
evaluation of early childhood programs for educationally disadvantaged 
children. Separately, each boo1< focuses on a single set of problems. 

The series include the following: 

. • Evaluating Title I- Early Childhood Programs: An Overview 
Assessment in Early Childhood Education 

• Short-Term Impact Evaluations of Early Chi^hood Title I Programs 

• An Introduction to the Value-Added Model 

• Evaluation for Improving Early Childhood Title I Programs 

• Longitudinal Evaluation Systems for Early Childhood Title I 
Programs 

• Evaluating Title I Education Programs That Involve Parents' 



The resource books are not comprehensive technical manuals. Their 
purpose is to help local school personnel clarify their information needs 
and decide what evaluation strategies are most appropriate to meet those 
needs. Additional information and technical assistance in using the var- 
ious evaluation strategies kre available in the more technical publications 
cited at the end of ea^ch volume and from the Technical Assistance Centers 
(TACs) in the ten national regions, w 

These resource boots and the evaluatidn approaches they describe do 
not constitute a sing^le evaluation system. Instead they represent a variety 
of approaches to evaluation aimed at serving a range of information needs 
at the federal, state, and local levels.' This strategy has been adopted 
because our study of the feasibility of^ developing evaluation systems for 
ECT-I programs revealed that the variety of p9tential uses of information 
at these three levels is so great as to defy any simple, concise or unified 
"answer.'* No single evaluation system, no matter how complex, could pos-' 
sibly satisfy all possible information needs (Bryk, Apling, 5 Mathews, 1978, 
p. 22-27). Nevertheless, we hope that the variety of ECT-I evaluatioh 
resource booklets which we have produce'd will help people to connect dif- 
ferent evaluation strategies to the varieties of information neeQed to 
improve and strengthen ECT-I programs. 
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I. INTRODUCTION 

'This resource book is one of a series -designed to help directors 
of early childhoo-. Title I projects evaluate and improve their programs. 
Taken together, the series addresses a-^riety of evaluation problems that 
are unique to early childhood programs for disadvantaged children. This 
particular book, however, may be useful for other purposes as well. The 
topics we discuss in this, resource book include not only how to assess 
the lt)ng-term impact of your programs, but also how to develop and use 
a longitudinal information system for a variet/ of routine operajtions. 
This dual purpose came about gradually, as the p^an for -the book evolved.. 
While our original goal was to describe methods for assessing the long- . 
term impact, or "sustaiiied gains," of ea/ly childhood projects, we soon 
realized -that a major part of such long-term evaluations involved develop- 
ing and maintaining^ longitudinal data base. Soon after that, we' realized 
that such- a data base could be useful for a variety of different purposes, 
both within and outside of early childhood Title I programs. Since the 
development and maint-enance of such' a system depends on your particular 
needs an^ ytfur particular situation, we have designed the book to help 
you decide what your particular information system should look like. But 
once you have decided that, you may want to consult a management expert, '* 
a consultant from" your regional Title I Technical Assistance Center ' (TAC) , 
or other, more technical book^, to help you get the system going. * * 

What exactly is a l^ng'itudinaJ information system? The term "longi- 
tudinal" has two meanings. First, it can'refer to the length of time, 
records are kept oj\ individual children; that is, you may maintain all*" - 



data on a given child from the time he, or she enters s<!thool until he or she 
leaved sixth grade. Such a rfecording system would be called "longitudinal. 
*The second meaning relates to historical information. You may, for example 
, maintain data on the characteristics of all kindergarten classes over a ten 
year pefiod. These data are also "longitudinal.'' The system you chofte to 
(Jevelop could reflect either (or both) of these definitions. But the mare 
impprtanf term is the word "system." For longitudinal data to be useful, 
' the^ must be collected and maintained systematically. That means: 

• Data collection is done at regular intervals 

• The population oti whom data are collected remains the same 
(i.e., "all kindergarteners," "all grade levels of a particular 
group," etc.)- 

• The data are filed in such a way that they can be easily 
obtained when they are needed. 

^ch a systematic longitudinal data base may not be possible in all situa- 
t^ions. For example, if your population is, highly transient, so that you 
wind up serving different children every year, you may feel that the 

^effort involved in creating such a system UiW not pay off. Or, if ynw have 
-a very tight budget, and may have to lay off staff in the near* future, 
you may jiot be able -to follow thraugh«on a system long enough for 
\t to pay ofh If you feel thesq kinds of problems will not deter 
you, read on. * - \^ 

This resource book contains four chapters. In the first chapter, 
we elaborate on the two ways in which longitudinal information systems 
can be used: for assessing long-term program impact, and for helping 
with routine operations. Ifi Chapter II, the two main types of such sys- 
tems are discussed, together with their strengths and weakncs.'^es . Chapter 
III describes hdw to determine your information needs, and Chapter V' 
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discusses the different design features of information systems that must 
be anticipated before one can be built. You may want to combine this 
book with information from other ^fburce^s; such as the Department of. 
Education's Title I Policy Manual, or materials prepared by your regional 
Technical Assistance Center. 
* ASSESSING LONG-TERM PROGFWr IMPACT 

Many, educators believe that if children's long-term academic or 
social, progress is to be .truly influenced, early assistance is necessary. 
If this is the reason for establishing programs for young children/then 
it is as important to assess the long-term effect^f the programs as, 
the short-term. Figure 1 summarises the evali^kion Questions" that^might 
be asked to asllss these kinds of long-term impact 
A Foundation for Future Learning 

"Early childhood Title I programs (ECT-I) may enable children to 
acquire certain attitudes or skills that are needed in order to attadn 
other skills. If so, the real effects of th'e progr^ will be seen in 
the attainment of the later skills. For example, an early program may 
be designed to foster positive attitudes toward education, so that . 
children will be more motivated throughout the|r school- years to study 
and learn. While early evaluation may show how the program affects 
children's attitudes, only later evaluation can determine whether 
those attitudes lead to acacfemic ^mprovemeht . Bilingual pro'^rams 
also illustrate the need to assess achievements that cannot be measured 
until some time after the program ends. The eventual' goal of many 
bilingual programs may be to help children speak, read, and write 
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^Long-Term Goals * 



Providing a 
Foundation for 
Future Learning 



Prevent ion x)f 
Future Problems 



Sustained Gains 




♦Sample Questions^ 

7^ '-. ' 

• Did the program Have an immediate impact on attitudes?x Did children 

• whose attitudes improved make' later achievement gains? 

• Did the program have an immediate impact On bilingual children's first 
language skills? Did children whose firs^-language skills improved 
later improve in English skills? 



• r-{6w many ECT-t "children returned to Title T later on? ^ 

• ilow many ECT-I chi;idren were retained in gra*de Later on? ^ . 

• How many."ECT-I children wer6 referred to special education later on? 



« If the program demonstrated no noticeable effects at its close, could 
Sleeper Effects children's scores still he affected in, later grades? 



' V # To what extent are effects apparent in the spring still visible the 
following fain* 

• Are effetlr-s still- apparent at the close of the following school year? 
'^"^JFhrough several schooT years? * J) 

• Did children 'who had two years of the program sustain their gains 
longer than children who had one year? , > , 
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ThQ 1978 Title I Amendmenljf require tljat .evaluations be carried out ''over at least' a twelve-month period 
in order to determine^ whether rogular school ybar projirams have sustained effects over the summer'' 
(Sec. 124(ji)(2)). These studies are to ♦be carried out ^t least once every three years. 

' _ ' Figure 1: Four Assessments of long-term Impact of Early Childhood- Programs 
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easily in both their first and second^ languages; but if facility with the 
first language precedes instruction in Jhe second language,^ a short-term 
impact study may only demonstrate ypur success in the first language. 



AgaiTi, to evaluate the full impact of bilingual ECT-I programs ,"lat^- ^ 
assessment is necessary, ' ■ ' 



r 

Prevention of Future Problems 



' , A frequent goalpf early childhood programs is to prevent later prob- 
lems in school--such as truancy, grade retention, or even droppong out of 
'high school--or the need for later services, such as special education 
services or more Title I services. ECT-I programs with the goal, of pre- * 
Venting later problems may not differ in content (such as attitudes' toward 
^ learning or facility with another language) from loiter program^, but are 
designed to give children an early start so that they can keep up with 

s . 4 

other children^^ Assessing the extent to which later^p^rf^rmance is free 
of problems is clearly far beyond the time frame of early childhood pro- 
grams, and therefore requires longitudinal evaluation, . ^ ^ 

Sleeper Effects 

Sometimes^ early childhood progra\rts have sleeper effects; that is, 
children may appear* not to have benefited from the program, Mit the pro- 
gram may in fact facilitate their academic; success' several years hence, 
^n eva*luation that stop3 at t^ end of the early program may thus lead to 
the erroneous conclusion that the program was ineffective, Vihy and when 
^ sleeper effects occur is conjectural,, but clearly, if they exi\t, longitu- 
dinal evaluation is needeJ^to find them. 
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Sustained Gains " * / ^ 

While someearly childhood programs show strong effects at the end of 
the program, these may fade out over time. E^^ntually, the children may 
appear to, be no better off for the program, and may ev^n fall behind again 
as if they had never taken part in it. Finding out whether the apparent 
early benefits fade over time is as important as finding out whether sleepei 
effects occur. In fact,- the Cpngress recognized the . importance of investi- 
gating whether gains were sustained when they revised the law in 1978. The 

i regulations now require tha^ at least once every three years, each project 
measure children's progress ovei> a twelve-month period. If the early 
benefits fade, cha:nges need to be-made. Longitudinal evaluation may help 
•determine whether early effects last long enough to, justify their costs. . 
ROUTINE OPERATION'S ' , - ■ - • . ' 

'The above aiscussion points out the value of longitudinal information 
systems for assessing the long-term impact of the program. We now turn 
to the ways in which good information can contribute to" a wide range of 
decisions routinely made in schools. 

Longitudinal information /"by definition, tal.es time, but it may still 
^ be useful for this year's decisions, and for future decisions, including^ 
some that cannot be anticipate^i^now. Because the potential uses are dif- 
ficqlt to define in. advance, and because many of them may be idiosyncratic 
to particular schooT systems, we confine our discussion to four major kind^^ 
of^use, rather than specifying particular uses. -Figure-: summarizes these 

* categories and gives examples of the Kinds of questions that might be 
asked in each. Let us consider nov. hou loneitudinal informat ion .might 
meet some of these needs. 



Routine Uses of 
•Longitudinal Information 



Sample Questions^'^' 



Programming 
Decisions 



Policy 
Development 




•Bstablishing 
Program 

Account^! 11 ty 



Decisions About 
Tndl vidua! 
Children • > 



• Which beginning readers prepare children best for their later 
reading texts? 

• Do m-claj^ services or pull-out programs have more "staying power"? 

• How does early at^lity grouping affect children later on? 

• At what age should early education programs be offered? 

• What eligibility criteria have the best long-term predictive 
power? . ■ . . ^ ■ 

• Should Title I programs be offered at all ages or are they mere' 
effective at one age than another? * 




. no children who leave Title I^e-enter the program later on? Do they enter 
enter special education? ^ ^i. uu tney enter 

t 

•mat is the.academic progress of Jitle I children after they lea^e the^program' 
*'skip°g1aSefr ' ^^"d-^^-tained in grade? 'how often do they 

•'Dormer Title I students graduate from high scho'ol? 
• Are former Title I students involved in extra-:curricular activities? ■ 
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• Should this child be referred to special education? 

• Should this child be taught by a different method? 

• Should we involve the child 's'parents more than we h3Ve? 
L V % 
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Figure 2; Routine Uses of Longitudinal Informal 



Programming Decisions 

This category refers to decisions about the content or organization 
of instruction, which for the most part affect what happens inside 
the classrooms. They include decisions about curricula, teaching methods, 
and grouping strategies. 

If early ,chi Idhood programs are to be improved, you must know what 
they are like currently. Information about the nature and intensity of 
Title I services provided each year, tor example, offers a starting point 
for analysis. Since class sizes and Title I budgets may change from year 
to year, the intensity of services offered to children may also change. 
Even within one year, services vary from classroom to classroom or building 
to building. Accurate data on what is available can contribute to dis-^ 
-cussions about more equitable or more efficient distribution of resources 
and to decisions about changing the general structure of the program. 

One sthool. district, for example, disc^overed »that -because staffing 
patterns had remained relatively stable for several years whi l^e the popu- 
lation of needy families h^d relocated, the case loads for social workers 
> 

were quite uneven. WTiereas one. school had 'three social workers^ for 100 
children, another had only one. These inequities appeared only wheri an 
information system was developed. Similar disorep^ncies may be found in 
the availability of teacher aides, particul^ly of unpaid parent volunteer? 

Comparisons across buildings may help pinpoint other imbalances--say 
variations in parent involvement. IsTiiie such information doe? not suggest 
a remedy, it at 'least identifies an area for prog/am ijnprovement efforts. 
Furt)fer investigations may suggest ways to increase parent inv(>l vement • 
changing the times when the Parent Advisory Council norm<jll> mect^, to 
avoid conflict with a popular neighborhood acti\ity; en<^ur ini;, thot 
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form-letter invitatiofts in a non-English-speaking neighborhood are not ^ 

<« — 

sent in English, ^nd the like. 

hTien information on services is cgmbined with information on 
children's academic progress, comparisons across buildings may contribute • 
to understanding the relative benefits of different service delivery sys- 
tems. Such comparisons must be made with care, however, since a variety 

> 

of other events also contribute to children ''s achievement. 

Information qn services that is tied to financial or staffing data 
can give a fairly complete description of the j^rogram, and thus suggest* 
areas where costs could be reduced without influencing the services pro- 
vided, or where resources could be more fairly distributed. Of course*, 
if children in different classrooms or buildings have different nelds'for 
services, this must be taken into account. 

Finally, full descriptions of the service? provided at each grade will 
show the extent to whidi successive educational e-xpeo-iences build on earlier 
ones, rather tha.n. repeating lessons, skipping lessons, or haphazardly' 
changing formatsi The data can indicate, for example, that some children 
received the same instruction in first grade as in kindergarten, simply 
because the first-grade teaeher did not knou what was offered in the Tatle 
I kindergarten program. Or they may suggest that some children have maved 
from highly structured programs to open classrooms (or the rexerse], 
simply because of anomalies in the placement process. Knowledge of 
continuity between Title I and^bther programs may be especially useful 
in designing Title 1 programs to prepare children for their later educa- 
tional experiences. 
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' There i^fa variety of ways, then, in which full and accurate informa- 
tion can as/ist in program planning. And longitudinal information adds 
the dimension of time to the possibilities for analysis^ Variations in 
services offered over the years may, for example, reveal gradual shifts 
in program emphfisis or in parent participation that otherwise might not 
be noticed. Good ideas put into practice by someone who has since Jeft , the 



staff might be forgotten if not recorded. Longitudinal' data provide an 
/ 

"institutional memorv" that extends beyond the memories of staff members 
who come and go. 

4^ Policy Developmen t ^ ^ 

IVhereas we have used the term programming decisions for decisions about 
how best to offer services to children, we use the term policy development 
to refer to a^lVother decisions about wbo should receive the services, what 
ages should they be served, wh"at should be the eligibility criteria, and 
so on. Longitudinal information would not only provide useful descriptions , 
of the student body, but might also demonstrate the relative success or 
lack of success of prior policies* Indeed, an institutional memory 
may be even more useful In tlie. policy sphere than in program planning. 
Policies must be continually revised to meet changing social values, chang- 
ing bureaucratic or budgetary constraints, and changing community needs. 
At the same time, the ways in which they affect children must be under- 
stood if policies are' to be improved. Without a^i institutional memory, 
a policy once adopted and rejected as inappropriate may be adopted and 
rejected again, for the same reasons. Suppose, for example, that <-i certain 
fest isJuseiJ to^determine children's eligibility for service^, bccau'^c 
it seems to be especially appropriate for young children and' is ca?> to 
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administer; but that it is later discovered that the test missed 'several 
needy children. Four years later, a new program director might try the same 
test again if he or she has no access to ^formation regarding its earlier use. 

Institutional memory a^art, information on the later academic success 
of children who took part in early childhood Title I programs is of great 
importance for judging progl^am success. Consider",, for example, the staff 
in one school district who thought their early childhood program quite 
successful until they learned that their third graders were having serious 
reading problems. Because the district had a relatively -transient popula"- 
tion, and because there were no' longitudinal records 'on the^ children who had 
been in the early childhood program, the staff did not know which third graders 
had participated in the program andtvhich had not. Longitudinal records of 
children's progress, then, could guide policy makers who must decide such 
matters as whether funds would be better spent in early childhood or in 
late^ programs. 

Since differences among groups of children coulff stem from many causes 
other than early childhood programs, information about these differences' 

/ 

can be useful for policy development. A technique of-ten used by insurance 

companies is that of developing actuarial tables. No causal inferences 

atej)veTtly drawn from the tables, but they are nevertheless used to 

establish r^te policies. In the case of automobile insurance, for example , 

accident rates are maintained separately for males and females, persons ' ^ 

under 25 and over 25^ and so on. In-a» similar vein, actuarial tables 

may be useful to educators. For example, if truancy rates vary by sex or • 

neighborhood, that fact may help in interpreting the "apparent success or 

lack of success of programs. Similarly, tables- of' a\'eragc reading achi ^-.vement 

* / .' 

a - P-* 
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at each grade, for children with different language or family-income back- 
grounds, may suggest what policy makers can reasonably expect as ''baseline" 
performance. These tables function essentially like local norm groups, 
but as many different categories of students can be developed as are 
needed to fit your local population. Longitudinal information can be 

especially, useful for infrequent events such as grade skips or retentions. 
The system allows^ data to accumulate oi^er a^ long time, until enough different 
children have skipped grades or been retained in ptades that ^atterrs can be seen 
Program Accountability , , , 

We use the term Accountability to refer to situations in Which <Jata 
may not lead to particular clecision§, h§it are nonetheless useful in justi- 
fying the program to parents, the community at large, or the fund^ing . * 
agency. Both immediate and long-term -program effects are an important 
part of accountability. Longitudinal information suggesting that program 
-participants fliaintain ^heir academic progress relative to their non-dis- 
advantaged peers" provides strong evidence of program success. And an 
information^ystem c^n contain much more than test scoi;^s: it can 
also show how often children enter and leave Title I programs throughout, 
tbeir acad^ic career, how many of them are retained in grade or referred to 
special education programs at some point, and how many have. histories of 
truancy or of exceptional academic progress. 

KTien the program goal is to prevent the need for later services, 
these data are valuable evidence of program effects.* If the evidence 
suggests that success has not- been significant, the- 'data may be used to 



* Provided that the criteria for receiving those^ services have not ch^ingct! 
over time. 
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revise policies regarding when and to whom services should be provided. 
meii goals are framed in terms of atihievement , longitudinal information 
can indicate variations in test scores across grade levels, across subjects, 
or across children who have entered school at different ages. 

The data can be summarized in various ways, depending on the' audience. 
Parents, for example, may be more interested in truancy and academic prog- 
ress, while funding agencies may want to know how frequently children who 
have been in the "early childhood program return to Title I later on. Annual 
progress may fe summarized for children with and without the early child-' 
ho^d program, children with one year or two years of the program, Spanish'- 
speaking versus English-speaking\^hi Idren, and the like. 
Decisions about individual Children 

"^s^ Teachers, guidance counselors, and other staff nfemb>rs make daily 
decisions about individual children: to try different instructional strate- 
gies or seating arrangements, to retain children in grade, or to refer ' 
t\)em to special education. Often such decisions are based primarily on ' 
the staff member's, own immediate knowledge of the'child, with only sketchy 
information on the child's previous educational experience^. Knowledge of 
which programs have been tried, whether the child has exhibited^^cademic, 
behavioral, or truancy problems in the past, or what the academic "success 
of his or her siblings has been may ass^ist in these decisions. ' 

Decisions about individual children are not confined to teachers. 
One school district recently^et up descriptive files of all children 
receiving special education and discovered that several of them had IQ 
scores in the 90s or even over 100! Since the scores were from tests 
given several y^ars earlier, the placement teams. had not known about them 

* a ' 
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when they placed the children. The data allowed fhe program director to 
identify children who needed to be re-evaltpated. 
PROBLEMS IN INTERPRETING LONGITUDINAL INFORmTION 

The vahie of early childhood Title I educational experiences lies 
in tlieir connection to.^J|^ter experiences, so much so that it would be 
difficult to separate the effects of one fr^jm/the other. How, then, can 
one estimate the contribution of the early experiences? Each child's 
education consists of , continuous experiences, and his or he*r success at 
any given time reflects the influence of the whole series. This fact 
makes it ciifficult to in^^rpret children's test scores. 

There are other sources of difficulty in interpretation. One of these 
relates to the tests themselves. hTiile the desire for longitudinal informa 
tion may stem ^ theory of child development, or a recognition of the 

relationship between early and later experiences, available tests .may not 
have been developed on that basis. Their content may change from grade to 
grade to accommodate naturally occurring curriculum ^changes across grades, 
but their score scales are rarely flevejoped by actually testing the same 
group of children over time. Instead, the first-grade test is standard- 
ized on first-grade children, the second-grade test is simultaneously 
Standardized on second graders, and so on. If, for some reason, feucr 
Title I children were included in the norming sample? in one- grade than in 
others, the scores of your Title I children relative to the "norm" may 
appear to change suddenly. Such changes would be due not to the actual 

educational experiences of your children, but to change m the group to 
« * 

whom they are being Compared. Tests must therefore be seUnted not onTy 
to match the curriculum over the several years of the study, but also 
with an eye toward hou changes in scores will be interpreted. 
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Int-erpretat,ion of .test scores may also be confusing becaijse 
^ events have^inf luehced children's attainment which were not T5art of 
the planned educational experiences, or because of historical changes 
in the- programs themse^lves which are forgotten and therefore do not 

enter into the. interpretation o^ children's later attainment. Longitudi- 

^ • J ^ ' 

nal information is oft^n interpreted under the assumption that the se- 
quence of programs experienced by each wave of children is essential'ly 
the same. ^In fact, class srze^ change iN^ith rises .and declines in schpol 
enrollment; textbooks change as new editions^ come out; teaching styles 
change as teachers manure or change jobs; and the children them^selves 
change over time, as their mothers^ do or do nof^work outsiide the^ome^ and 
as otlier influences, suc^l^ as television programming, change. ^Unles^ your 
longitudinal information system somehow takes historical events'^ into 
account, the data may be erroneously interpreted. ^ . 

Finally, since children may enter and leave the Title I^^gram in the 
course of their education, and since many Avill leave t^he school system 
altogether while othei^s transfer ^into your schools from other districts, 
■ilTiformatiort neglecting these changes'may be difficult to interpret. 
SUMMARY 

This chapter has described in broad terms' the two ml^in uses of lon^^i- 
tudinal information systems: assessing the long-term impact of early 
•childhood programs, and contributing to routine decisions. In each case, 
the variety of suggested uses implies that such information system? would 
be valuable. The applications of longitudinal inforraatioji arc summ;irized ' 
in Figures 1 and 2. We have also seen that for <i6veral rea'^ons the informa- 
tion may be difficult to. interpret . Thel^maindcr of this resource booK 
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will be demoted to more pragmatic concerns of develoj3ing and using an 
j^nformation system. 
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• II. TWO KINDS OF LONGITUDINAL INFORMATION SYSTEW 

There are two general strategies for developing longitudinal informa- 
tion systems. Each has its own strengths and weaknesses , and the two 
differ in their appropriateness to. different situations. For convenience, 
we label these two strategies controlled studies and descriptive studies,, 
to reflect one of the main differences in the two approaches: control. 
Whereas the first strategy entails care&n corfJrol over which children 
receive which educati-onal experiences/ the second allows children to, be 
- assigned to programs'un routine ways. There are other differences, too. 
"Control" mqans that children are assigned to different programining arrange- 
ments systematically, so that specific questions such as "what "^uld hap- 
pen if. . ." can be answered. Description, on the other hand, does not 
entail such controls, and so cannot provide specific^^answers; but it may 
provide useful insights into a number of different questions. And this 
leads to a third difference in the two strategies: the descriptive 
* system, in its relatively broad ^cope and flexibility, is more appropri- 
ate for exploration, whereas the controlled study of groups permits strong 
inferences to be drawn^ from the data analyses. That is, the latter 
frees you from nagging questions. about altkqiative explanations for observed 
differences among gaw^aps. With these differences clearly in mind, let 
us consider eacK approach in more detail. 
CONTROLLED STUDIES 

^_ControlIed studies allow ditect comparisons across groups'of children 
who have had ''different educational experiences. For example, one group 
n>ay follow a^ DJ STAR ^ program while another uses, say , 'Sul I'ivan readers.. Or 
one group may take part in preschool programs at ages three and four, another 
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only at age four, and a. third 15^ Until kindergarten. The comparisons 
of interest must be specified ahead of time 50 that services and children 
can be arranged to allow comparison. 

Figure 3 displays several aspects of an ideal controlled study of 
a one-year program. First, a population is identif ied--children eligible 
for an early childhood Title I program, for exam^Jle. Next, a sample is 
randomly selected from this population and randomly assigned to a program 
group or a comparison group. These groups may participate in two different 

kinds of program; or if funds are scarce, one group may take part m the 
program while the comparison group does not. Each group can be tested 
before the program begins (time 1), but need not be. At the end of the 
program (time 2), all groups are tested, and testing continues for alj 
groups over a period of time (time 3,4,...) as children continue 
through school. Finally, those children who leave the program early, for 
any reason are followed over time and tested at the same specified inter- 
vals. This is crucial to later interpretation of the data. If the drop- 
outs are random, they are not likely to cause any differences later 
between program and comparison groups. If they are somehovv dilffefent from 
those who stay (for example, tend to come from poorer £||dlies), th'en 
comparisons of the remaining groups may \>e difficult to interpret . ^ 
If the program and comparison group members are^^pared at times . 
1, 2, 3, ... , the immediate and long-term program effects can be 

* It is highly desirable to have a series of iTieasures for which results 
can be expressed in *the same »way, such as in percentile ranks. That 
is, the same or comparable tests or measurement techniques should be 
used throughout the study. These measures should be able to show ^^rowth 
aciv ^ss th e years, and if possible, relative standing at^ any given time 



Oq 



Population 
of 
Interest 



Random - 
Selection 



Chi Idren 
to be Included 
in the Study 



^ Random 
Assignment 



Pretest 



Posttest 



Fol low-up 

Test 



Group A - - 



Group B 



Fol low -up 
Test 



Program A [- - _ 



Dropouts 



Program B 



Dropouts 



TiiB^' 1 



Regular" 
Program 



Dropouts 



^ Regular 
Program 



Dropouts 



Time 2 



Regular 
Program 



Dronouts 



Regular 
Program 



Dropouts 



,Time 3 



Time 4 



Figure 3: An Id^al Controlled Study of n One-Year Program ^ 
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examined. In addition, comparing immediate and long-term effects will 
demonstrate whether short-term gains are sustained and whether long-term 
sleeper effects have occurred. The ^udfence is strong because of the 

control over the study; the program experience should be the only difference 
between final program group ancl. comparison ^roup. Thu§, any gains of the pro- 
gram group vis-a-vis the comparison group must be due to the program. 
An Example i \^ 

Louise Miller and Jean Dyer (1975) compared four different Head Start 
programs as they affected disadvantaged children. Four-year-olds were 
randomly assigned to the different programs and attended them for one 
year. Miller and Dyer added a group of co^nparison children who had no 
He^ Start experience. Head Start children were given a battery of tests 
when they entered the preschool programs, and they and the comparison 
children were regCITarly tested from entry to kindergarten through the 
end, of the second grade. 

In addition to randomly assigning children to programs, these evalu- 
ators took care to equate their groups on a number of other conditions, 
such as the kind of grade-school programs children went into after Head 
Start, and the general quality of the buildings in the schools the 
children attended. They also videotaped the trlasses, not only in Head Start 
but in kindergarten and first grade as well, to be sure that the programs 
were carried out as they had been planned. Clearly, this example of a 
controlled gilfiup comparison is one of 'considerable complexity, thoroughness, 
and cost. Just as clearly, the care paid off in complete and interpretahl^ 
findings. 
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If controlled comparisons of groups can be done, they can provide 
strong and convincing evidence of both short- and long-term program 
effects, Nb doubt, however, you are already listing r^sons why this 
strategy is not practical. In addition to the problem'of costs, find- 
ing and maintaining a c(Tmparison group may be difficult if not impossible. 
. Randomly assigning children eligible for Title I to program axid non- 
program groups may not be legal, since the programs are mandated to serve 
the nee4iest children. Some program directors may be genuinely interested 
in the relative benefits of two alternative programs, however, and for them 
controlled comparisons may be possible^ since 'each program can ser\'e as a 
comparison for the other. But for most, the Question of real interest 
is "early program versus no early pi»gram''--a comparison that is almost 
impossible to make. by randomly assigning children to groups. Moreover, 
locating any non-program comparison group, especially for prekindergar- 
ten programs, can be difficult, because parents will not be eager to 
allovv testing and observation of their children without receiving program 
benefits. Although this is not as severe a problem with kindergarten or 
first-grade programs, for which there is an available pool of children 
not in Title I programs, random assignment of those eligible for Title 
I may still not be possible. And children who were not eligible for Title * 
I will provide no clear evidenci of how thotse who were woulti^have done 
over 'time in the absence of the Title I program. Finally, if the compari- 
son group is in school, it cannot be said that it follows no program, 
for many of its members may be in other compensatory ^programs » some quite 
sin^ilar to Title I. There is ri ne6d, then, to define vshat Kind of educa- 
tional program, can be considered a '*no-progran." experierue that Vould he 
meaningful to compare to the Title i experieiuc. 
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Attrition may also make controlled studies less feasible. If the 

attrition rate differs for the program and comparison groups, and if the 
difference cannot be attributed to chance, then observed differences in 
outcomes may be due to differences in the children rather than the pro- 
grams. If loss is random--that is, if the final sample 'is not signifi- 
cantly different from the dropout5--attrition is only a technical problem. 
DESCRIPTIVE STUDIES 

In many cases, a simpler descriptive approach more readily fits 
the realities of early childhood Title I programs. Since ECT-I children 
are sel^ted because of educational need> random assignment, as we have 
said, is generally impractical, if not illegal. Descriptive systems do . 
not require random selection or assignment of children to programs; they 
just follow up whoever was selected. In addition, using the same or 
comparable measures oveir time may not be possible, given the range of 
program goals for different age groups. Descriptive systems do not re- 



quire the'same or comparable measures. Finally, the descriptive system 
can portray the actual educational programs that children take part in over 
time, and so provide a complete knowledge of both program experiences and 
academic progress. ^ 

Because systems can.be designed in different ways to meet different 
information needs, there is no "ideal" descriptive system. Figure 4 
indicates the groups on whom data could be collected if the system spanned 
several school years (grade k through 6j . Even more groups could be 
defined, since each wave of children may contain several different sub- 
groups: EngUsh- or Spanish-speaking;, children with or without Head Start 
experience; children with or without earl> childhood Title 1 programs; 
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Figure 4: Different Groups of Children in a Longitudinal 
Descriptive Information System 
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children who did or did not drop out of-'those p^j^rams, and so forth. 

y 

fhe data maintained could also include achievement test scores, teachef 
ratings, grades, attendance, assignments to special education, grade 
retentions, or re-entries to other Title I programs. 
' An Example ' , ■ - ^ ^ 

Lazar" et al. (1977) analyzed the descriptive information available ^ 
on a project in Vineland, New Jersey. Their analysis shows how a com- 
parison group can easily be added to a longitudinal descriptive informa- 
tion system, and illustrates the usefulne^s^f j^rogram outcome measures 
such as assignment to special education and grade retention. The partici 
pants "in this program, much like ECT-I children,, were the highest-risk 
children; thus no similar group could have been found for comparison. . 
The researchers therefore decided to compare the children in the program 
to the general school population, represented by a random sample from the 
previous yearns first-grade classes. This sample, of course, included 
middle-class as well as lower-class children. To compensate for the 
higher percentage of Spanish-surnamed children in the program group than 
in the general school population, 36^ additional Spanish-surnamed children 
were selected randomly from the previous first-grad^ group. This, the re 
searchers claimed, compensated* for ethnic and social class^ differences. 

Two comparisons were then made. First, the program children w^rc 
"compared to the random sample of the general school populatiorf, exclud- 
ing the 36 Spanish-surnamed children. Then, the Spalii sh-surnamed program 
children were comparec^ to the sample of bpanishrsurnamed children in 
tWe general school population, including both those in the first sample 
and the 36 additio^ial children Cp. 701. 
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Tables 1 and 2 illustrate group differences in grade retention and 
in assignment to special education. Since statistical tests assume ran- 
dom assignment, they are not appropriate for these comparisons; but fre- 
quencies and percentages alone can be useful. For example, 'Spanish-sur- 
named children are retained in grades at a higher^rate than other chil- 
dren; this seems to warrant further investigation. And program children 
are more likely to be assigned to special classes. This could mean that 
they need further help, or that there is bias in the special-education 
assignment system. 

The descriptive longitudina^information system clearly differs 
from the controlled strategy described earlier, and therefore cannot 
be expected to provide the same amount |^ kinds of information. Most 
significantly, the descriptive study cannot be' used to infer that ob- 
served differences in children's accomplishments were caused by the pror 
gram, since too many other contributions are left uncontrolled. But 
it can, for example, -describe the conditions of ECT-I graduates. Do 
these children remain in Title I programs? Do they leave Title I only 
to be cycled back in later? Do they end up ip other compensatory pro- 
grams? Or. do they enter and continue through regula-r education programs? 
Descriptive information can also tell you how many of your second- or 
third-grader^ participated in the eaxly childhood program, how many of 
them transferred into your school district, and even how often children's 
absences contribute to missing data. 

But the descriptive study, like the controlled study, has its own 
difficulties and requires a substantial commitment of time and skills, 
as well as continfjity of -staff for maintenance. In addition* since its 
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The Effects of the Micro-Social*Learning System on Grade Failure 



Program Children vs. General School P6pulation 



Not Retained 



Retained 



Total 



Program N 

% of Program 

General Population N 
% of General 
Population 

Total 



132 
(68. 0%)- 

66 

(65.3%) 

198 



62 

(32.0%) 

35 

(34.7%) 

97 



194 

(100%) 

101 

(100%) 
295 



V 



Spanish-Sumamed Program Children vs. Spanish-Surnamed School Population 



Not Retained Retained Total 

Program N 80 ^9 119 

% of Program (67.2%) (32.8%) (100%) 

Population' N 23 39 62 

% of Population (37.1%) (62.9%) (100%) 

Total 103 78 181" 



(Source: ' Lazar, 1977, p. 80) 



This was tl^e name of the program developed for the children. 
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Table 2 



The Effects of the Micro-Social*Leanting System 
on Assignment to Special Education 



"Program Children vs. General 


School Population * 








Not in Special 
Education 


In Special 
Eduction 


Total 




Pr.ogram 

% of Program 


169 
(86.2%) 


27 
(13.8%) 


196 

,(100%)' 




General Population N 
% of General 
Population 


95 
(94.1%) 


6 

(5.9%) 


101 
(100%) 




Total 


264 


33 ■ 


297 • 





t 



Spanish- Sumained Program Children vs. Spanish-Sumamed School 


Population 1 




"Not in Special In Special 
Educa t i on Educa t i on 


r 

Total ^ ^ 


Program N 

% of Program 


104 17 
(86.)%) (14.0%) 


121 

K100%) 


Population N 

% of Population 


/ 54 8 ^ 
(87.1%) (12.^ 


62 
(100%) 




158 * 25 


183 

9 



(Source: Lazar, 1977, p. 81) 
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This was the name of the program developed for the children. 
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benefits derive from its ability to contribute to ongoing h>rpothesi,s 



testing and problem solving, it is only as good as the descriptive in'form- 
atlon It contains. • 
SUMMARY AND. COMPARISON 

We have outlined two different kinds of longitudinal information 
system, the controlled study and the descriptive study. Each has 
advantages and disadvantag'es. The controlled study requires great care 
in sele,cting the educational experiences that could meaningfully be com- 
pared with Title I experiences, and in manipulating children^s^^lacements 
so that the comparisons are valid whilj& the. Title I rules are honored. 
The descriptive study, on the other hand, -requires* maintenance of quite 
a bit more information, since you do not know in advance what kinds of 
comparison will be useful laterv on. It could eventually be mbre costly 
and more complex to manage. 

Generally speaking, the controlled study is better Suited to assess- 
ment of long-term benefits of early childhood programs, since it compares 
the achievement of chi Idren* with and without the program; whileVfie 
descriptive study is better suited for routine operations, slice it can 
provide information not only on outcomes, but also on other aspect^ of 
children and on program characteristics as well. But this distinction 
is not absolute* In the controlled study, several kinds of information 
need to be retained, including data on the characteristics of children 
who remain in the program and of those who leave, so the info^ation sys-^ ^ 
tem^developed maV turn out to be useful for other purposes as well. And , 
descriptivestudies may enhance your understanding of long-term program 
benefits, provided conclusions are carefully dr^wn. The next chapter 
discusses in greater detail how you can determine your own information need*^. 
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Ill / DEFINING YOUR INFOWIATION NEEDS 

Both types of longitudinaa information system described' in Chapter 
Two require a great deal* of planning if they are to be useful for the' 
purposes we have discussed. . Because such systems can easily become too' 
large to be manageable, one 'of the most important p'arts of planning is 
to decide exactly what information you will need. .This chapte.r provides 
a number of exercises to help you do that . T^^vare divided into three- 
stages: getting started, expanding, and" compromising. The two getting 
started exercises involve Seciding who will use your information system 
and determining how they will use it. l\'hen you finish these exercises, 
you should have a set of "wish lists" of information different people 
would like to have. The next stage consists of expanding those lists 
by considering all sorts of other information that you might, need to 
interpret the data on your original wish lists. By the time you finish 
this stage, your list of possible information should be quite long—too ' 
long, in fact. Then you need to start the third stage, which consists 
of compromising. To help you with your compromises, we provide a^ number 
of tables you can .use to list and compare all your possible types of 
info-nnation,{^s^that you can choosfe those that will be most feasible for 
you to collect . - ' 
GETTING STARTED 

Since information can be used by many people and in many, ways, the 
first step in developing your system is to determine who will be the 
primary^sers and what will be the primary uses. 
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Identifying Users ' 

Different people in the district have different information needs 
and may require information at different times. Guidance counselors, 
for example, are more likely to want data on ijidividual children than 
aggregates of data. Teachers may also want data on individual children, 
and occasionally* on class averages. Principals may be more interested 
in data on the children in their buildings than oif those in the ^ole 
district, while program directors and other district administrators may 
need aggregated data on all children. . 

It may not be possible to develop a single system that responds to 
all these needs. If data are stored longitudinaHy ,by child, aggregations 
across children may be more difficult; if they are stored b]^ grade level, 
it will be difficult to reconstruct a file on an individual child. 

The question of who will use the system influences more than just 

« 

the way the^nformation is organized. If more than one person has access 
to the files, safeguards for maintaining confidentiality must be built 
into the system, and each user must be trained in how the system is 
organized and how to find the information ^e or she needs. 

T^e fact that different people may waivt different kinds of information 
need not mean that the system must exclude some possible users. Yet attempt- 
ing to serve all potential u§ers may not be the best solution, since a compro- 
mise may mean that no one can easily get the information needed. One reason- 
able alternative may be to maintain two sets of fileS--one with longitu- 
dinal data on individual children, and another with aggregates of 
data maintained for making comparisons or reporting general statistics to 
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the parents and the conununity. is not such a monumental task as it 

might seem. The raw data (test booklets, teacher ratings, program enroll- 
ment data) are available anyway, and can easily be filed in one system, * 
while the scores from these data can be summarized in another system. 
Identifying Uses 

^ Earlier, we described four broad categories of use (policy development, 
accountability, program decisions, ancf decisions about individual children) 
for descriptive information systems, and suggested various kinds of comparison 
for which the systems are suitable,' Since different uses imply slight differ- 
ences in system organization and information to be collected, it might be use- 
ful to rank-order the comparisons in terms of the likelihood that^ou would want 
to make them, or in terms of the relative value they would have for you. 

One way to sort out uses is to review instances in which people wished 
they had ha4 more information. The following situations, for example, may 
have l^d to that wish. 

• A visit by a Title I program reVrew team , 
Preparation of the last Title I grant proposal 
A school board budget-making meeting 
Negotiations with the teachers* union 

• A visit from the state education agency 

• A conflict between the Title I director and the director of 
special education programs 

A local newspaper report on the school district 
ft » 

• A PAC debate • 

A decision on staff ot textbook changes. 
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It may be helpful to ask several people who were involved in these events 
*to list informaMon^hey wish they* had had, or, if they can, to outline tables 
showing hovt they would like to have seen it displayed. 

Once you have detewined who will be the primary users -of 
the inf#mation system, how they would use it, and what information tliey 
needed during recent critical events, you will already have developed a 
good-sized list of potentially useful information. But to make that infor- 
mation as useful as possible, you may need to expand it further. The next 
section discusses some ways to anticipate-the need for ancillary information. 

EXPANDING " 



\ Upon reviewing different "wish lists," you may discover that information 
is often wanted for comparisons such as those made when assessing long-term 
program impact. That is, people may want to know whether this eligibility 
critericjn picks up more needy childi*en th^ that one, whether Spanish- 
speaking children benefit from the program as much as English-speaking child- 
ren, or what the addition&T benefits of a, summer program are relative to the 
regular schoal-year program. 

But ^nle^s these comparisons are based on controlled studies, they 
' cannot lead to sound .conclusions about wh^ things look the way they do. For 
example, since many things can influence children's academic progress, 
differences between children who have participated in the early childhood 
program and those who have not 'cannot be attributed solely to the program, 
they may have been there eSrlier, or be due to post-program experiences. 
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For any comparison that can be made, there are several plausible explanations 
for observed differences. If you can anticipate comparisons you will want 
to make,' you can design your information system accordingly—to control 
children's assignments, or to include descriptive information that may help 
you interpret uncontrolled comparisons. Let's consider several kinds of 
comparison that could be made, to s«e how planning can make them more 
uleful. 

Program vs. No Program 

Suppose children who were in early childhood Title I programs are tested 
on reading comprehension' at the end of third grade, ?nd that the scores are-\. 
lower ythan those of children who have never had Title I services. Can we 
infer that the lower scores are due to the program, that children are bet- 
ter off without it? Certainly not, for the program children diffel^ed in 
several ways from the non-program children: 

• They wer^ behind academically when they first started school 

^ # Many of their families may be poorer, and may not be able to 
provide as many stimulating' experiences to their children as ' 
other families can i 

• They^y be more likely to have single parents and to be cared 
for during the day by their grandparents. 

If you know that you will want to make such a comparison, you can do 
one or both of two things in advance to help you interpret it: you\an control 
asftignments to the program, so that the two groups are more similar; and^^rou 
can collect and maintain information on the characteristics, of the two groups, 
so that you will knpw exactly how the groups differ. Since Title 1 service*; 
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must be provided to the neediest children, you cannot randomly assign chil- ^ 
dren to program and no-program groups. But you may be able to identify other ^ 
groups of children who are similar: poor children who are less educationally 
disadvantaged, or higher-income children 3|p other schools who are equally 
educationally disadvantaged. Neither group offers a perfect 'comparison, of 
course, so information about their/characteristics, as well as about those of 
the Title I children, should be maintained. The important poi^t here is to 
ideYitify the comparison group in advance, and to collect information about 
it in advance. ^ / 

Program vs. No Program: Title I Eligibility 

As data are being collected, it may become apparent that some of the 
children were eligible for the program, but for various administrative reasons 
were never identified or sen'ed. If you are operating a preschool program, 
"Kor example, your screening procedures may not have found all eligible child- 
V ren. Although this would normally occur only rarely, the existence of even a 
handful of such children may be enormously useful for comparison purposes, 
since their backgrounds are probably more like those of children in the program 
than are the backgrounds of children who are not. Such cjiildren may invite 
frequent comparisons and ready interpretation of obser\'ed differences, even 
though they are not really the same as those who took part in the program. One 
must ask why these children were overlooked initially. Are their families ncrc 
out of touch with the school? /lore leery of educators^ Do they live i/i 
different neighborhoods? Speak different languages" Though it ma> bo hard 
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to determine how served and eligible unserved children differ, these dif- 
ferences must be sought out, "for they.could influence the educational pro- 
cess as well. This information could also become a part of the longitudi- 
nal information system/ and may be useful not only for interpreting compari- 
sons between children in and outside the program, but-also for improving the 
process of identifying eligible chJldren 
Programs in Different School Buildings 

If the program is offered in more ^than one building, progra- results 
from one building to the next may be of inr^est. But children m different 
buildings may vary in many other ways. Each building may serve a different 
neighborhood, and each neighborhood may have its own cultural values and 
attitudes toward education. / 

Since children in all schools for which outcomes are to be compared arc 
receiving Title I services^ they can, in principle at least,, be randomly 
assigned to schools. But random assignment may not be practical, since 
it would ontaU busing children away fvom th?ir neighborhood schools. If, 
no special assignment procedures are used, comparisons of program effects 
in different schools should include the same information on background 
characteristics as do comparisons.of program vs. no-program. 

Furthermore, if~*^ comparison is of interest because of known dif- 
ferences in programs, then' you should also keep information on the nature of 
the different programs. Was the program in one building changed so th.it 
it would better meet the needs of children in that neighborhood'^ llou ^o^ 
Or were new materials simply placed there to try them out^ If so. uhy 
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in this building and not another? Is there a teacher there who advocated 
these materials? If so, the differences in outcomes may reflect the' teacher's 
zeal more than the effect of the materials. , Finally, the buildings themselves 
may j^iiffer--one has larger rooms, one is newer, one is designed for open 
classrooms while another has seats bolted to the floor, one has a library, 
and so forth. \ 

These slight differences bet^en neighborhoods, or between program 
characteristics from building to building, may be difficult to document and 

maintain. m ar\ information system, but they will prove to be enormously use- 

« 

ful to those who wish to interpret comparisons of child outcomes across 

J 

school buildings. 
Waves of Children 

If the program changes over time, or if it was only recently initiated, 
it may be useful to compare the academic progress of different waves of chil- 
dren, sinc^xyie>- will have had different program experiences. But just as 
^ other comparisons^can'be interpreted in several ways, so can comparisons 
of waves. Unless the di'strict has introduced a neu desegregation program, 
one wouldn't expect the population to change too radically from year to year, 
but other events could occur that would affect these children: 

• Local emplo>Tnent rates may rise or fall rapidly if a large 
company obtains or loses a major cqptract 

• A new public television station may bring educational pro- * 
grams such as Sesame Street and The Electric Company into 

range 

• A lengthy teacher strike may deprive different waves of ^ 
different jeducarional experiences 

• An historical event such as"a rocket launching or an elcc- ^ 
tion may alter curricula so that some waves arc taui^ht 

more current events and less standard curriculum than 
others. 
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Once again, these variofas^ influences may be difficult to aitticipate or to 
document; but , including such i;iformation in the system^ma^ be a great help 
in interpreting cross-wave comparisons. 
Age or Grade Level 

A major advantage of longitudinal information 'systems is that they allow 
children's academic progress to be tracked over time. Do the early program 
effects iast? Do scores rise temporarily, only to fall lateFSii? Of course, 
all children's scores go up over time, so progress is usually measured rela- 
tive to that of s,ome other group of children-children in your district, or chil- 
dren in j:he norming group on which the test scores were developed. Longitu- 
dinal questions, then, tend to be phrased as, "Kere the differences, between 
Titlf I children and t*he total ^chool^population larger at the end of first 
^grade than at the end of fourth grade- Interpretation of these comparisons 
requires just as much knob-ledge of the comparison groups of children as any 
other comparison does. If program children are being compared on first- 
grade versus fourth-grade standard scores in rjsading comprehension, then the 
characteristics of the norming sample should be compared with the character: 
istics of your' program children. What is their family, income? How many had 
only one pare'nt? Khat wave were they in, and how might that wave differ 
from that of your program children? 

In addition to differences between groups, differences in tests used 
at different times may also affect these comparisons.- For example, the 
first- and fourth-grade reading tests may differ on any of the fol levying: 

• Emphasis on word recogni^tion vs. sentence comprehension 

• Test reliabi lity 
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•*Time needed to take the test 



• Oral vs. written test-taking in/^truction 

• Match between content of test items and curriculum 



•>Qualif ications or experience of the test administrator . 
Since all of these, differences among tests couTd affect children's scores, 
they should be taken into account in interpreting cross-age or cross-grade 
' comparisons. This will be easier if the peculiarities of each test and 
each test administrator are documented and placed in the information system 
as tliey are met, so that they are not forgotten later when the comparisons ^/c 
made. 
Attrition 

If the community population is highly transient, you may need to know 
the nature and extent of attrition in your program's, as well as the charac- 
teristics of children who leave compared with those v,ho stay. That information 
can'be used for program planning, but, is also especially helpful when other 
comparisons are interpreted. Suppose you are looking at fifth-grade achi^eve- 
ment scores in schools with*and without early childhood Title I programs. 
How many of the children in title 1 schools actua^lly had the early childhood 
program? In a highly transient community, there ma> be few who did, and that 
is worth knowing. 
Summary 

Since we cannot list all the ancillary information that you might need 
for your population and your evaluation questions, wc have tried instead to 
provide examples of how such information may be usc^d to interpret" di fferent 
comparis6ns. You may easily think of other kinds-of information that you 
might need^to interpret comparison*^ of particular intcrc<^t ta your particular . 
Title 1 early childhood program. But you may also feol like you cnn*t 

ERIC • _ • 43 " ' • . 



■39- 



possibly collect all this information. The next section suggests ways to 
select those pieces of information that will be of most help to you. 
CWPROMISI.NG 

Once you have completed your starting and expanding exercises, 
you ftiay find that your list ^f potentially useful information is 
quite long. Tables 3 and 4 list a variTety of kinds of information 
that might be useful, but only as-examples. Your exercises may lead 
you to several others. Once you have your ideal "wish list," you need 
to find ways to pare it down to a list of things on which you can 
really collect data. 

\ 

The information people would like to have is often not easy to obtain. 
For example, one may wish to know how frequently children have trouble with 
school, whether that trouble is in their academic progress or in their atti- 
tudes toward education. The concept of "having trouble" is a broad one, but 
several indicators of trouble may be available: tardiness, truancy, referral 
to the principal's or counselorJs off ice. . grade retention, and referrals to 
special education,- are all indicators' of trouble) just as are test scores, 
grades, and behavior rating scales or checklists. Similarly, "success- 
may be measured by test scores, participation in extra-curricular activities, 
winning prizes, or being on the honor roll. 

h-hereas a review of recent situations in wh-ich information was needed 
may yield examples of ideal information, it is equally important to deter- 
mine the best indicators of each ideal. This is true even of more readily 
agreed upon concepts such as achievement, which could be measured by 
criterion-referenced tests, teacher-made tests, grades, pages completed 
in work-books, writing samples, and number of objectives mastoBed. 
Thus, two stops are needed: to define the idcil inrorn.it i on 
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Table 3 

Kinds of Student Information That Could Be Catalogued 
In a Longitudinal Information System 

Student Status 

• Age 

• Age at entry to school 

• Address 

• Parent income and education 

• Race* 

• Second language 

• Ffealth impairment (e.g., asthma, diabetes) 

• Number of adults in the home 

• Number of siblings; their ages, grades 

• Title Ueligibility, by year - 

• Presence in Title I programs, by year* 

• Whether student dropped out of Title I mid-year . 

• Test scores or grades, by year* 

• Teacher ratings, by year 

• Attendance 

• Grade retentions or skips 

• Presence in special education programs, by year 
t Extra-curricular activities, by year 

• Honors or awards 



Test Score Data 



• Content 

• Match between content ^nd curriculum 

• Test-taking skills required 

• Norm group average income 

• Norm gr6up cohort 

• Average family size of norm group 



r 
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Table 4 

Kinds of Program Information That Could Be Catalogued 
In a Longitudinal Information System 



r 



Title I Services 



• ClacsrooD 

• Curriculum, by year* • 

• Individual/small group/large group instruction 
^ Use of aides 

• Number of days/year, by year 

• Number of hours/week, ,by year 

• Class size 
••Mean class ability 

• Parent participation in class* 

• Parent attendance in PAC* \ 

• Parent attendance at teacher-parent meetings 

• Other special services Available 

• Ratio of service providers to children* 

• Unusual curriculum changes that reflect current 
events (e.g., a rocket launch, an assassination) 

• Unusual events that modified service delivery 
(e.g., a flood, a teacher strike) 



Finances 



• Cost of teachers 

• Cost of workbooks 

• Cost of consultants, itinerant 
teachers, etc. 

• Cost of special equipment 

• Cost of miscellaneous "materials 

• Cost of overhead 

• Cost of administration 



Staff 




• Teacher age, years experience, 
education 

• Aide age, years experience, 
education 

• Itinerant teacher age, years 
experience, education 



^Already required for Title I reports 
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and to define the- indicators that may be used to provide insight into the 
cqncepts of interest. Let us review some methods for selecting among the 
kinds of information listed in Tables 3 a"nd 4 that you. can afford to col- 
lect and maintain. 
Choosing Influences on Outcomes 

Earlier, we pointed out that descriptive studies of groups yield less 

certain evidence of program effects than controlled studies, and' that 

♦ ^ 
uncertainty Jifakes it nece^ary to maintain data on various non-program fac- 
tors that could infJu^nce the children's progress. Since many of these 
potential influences may be difficult to measure, it ^lay be useful to con- 
struct a table summarizing them and their contributions to comp'arisons you 
will want to make. Figure 5 illustrates such a table. Since the possible 
influences on childrlen^s academic progress are infinite, some rules are 
needed for inclusion in the table. We have chosen four kinds of influence:, 

# Those that school districts already tend to control 

# Those that school districts could control, "but tend not to* 

# Those related to the quality of the indicators 

# Those that school districts probably could not control. 
Whether these influences are or are not controllable by the school district 
should not affect the decision to maintain data on them, since the purpose of 
such files is to aid in interpreting comparisons. If t^ie children served 

by different schools come from significantly different family background? 

and there is reason to believe that this influences' academic progress, then ,^ 

• I 

the backgrounds are relevant to interpreting academic progress, whether or " 

I 

por the school can control them. j 
Your own knowledge of the children and the community you serve mny fead 
you to hypothesize other influences on children's academic or social pro- 
press in scfiool . You should list these as well. Once you have listed all 
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Influence 




CQntrolIed by the School * 

• Size < 

• .ronriculum 
Eligibility Policies ^ 

• Teacher Education 

• Presence of Waqher Aide 

• Teacher Race or^Language 

Could be Controlled by the 
School 



Family Attitudes Toward 
Education 

• Teacher Strikes 

• Distance From Home to 

• 'nfpnt Participation in 
School Decisions 

• Parent Tutoring' 

• 

Related to the Indicators 

. *# Test Anxiety 

• Reliability of Tests 
*# Match Between Test Content 

and Instruction 

• Non-substantive Recfui^re- 
ments (Child Patience,/^ 

\ Docility) 

.. (N^t Controllable by the 
• j^hool 

• Parent Income 

• Parent Education 

• First Language 

• Neighborhood Value Placed 
on Education 

• Presence of Older Siblings 
. • 'Numb^ of Adults in the 

Home 

• Pretest Scores 



Strength of Influence 
on Outcomes 



Availability 
of Datig^ 



Figure *5f Influen ces on Outconos : Intet^retjtj^njDf Comparisons 
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the influences, you need to identify those that you "consider the most imjtortant 
and that must, therefore be recorded in your information system.^ A; simple 
rating scale, where, say, a score of 1 means hi^ly influential ^and a score 
of 10 means not vprv influential at all, may suffice for choosing what you 
should'record. You may also "want to score these influences on^ a second scale 
that roughly indicates how easily accessible the information would be. 
Choosing the Indicators ' ' - . 

No indicator will perfectly measure a conc-epf of interest. Just as the 
Gross National Product imperfectly indicates the general economic health of 
the nation, so do reading scores imperfectly indicate the general academic 
progress of st-udents. But for any given concept, many indicators are 
possible--niore than can easily be maintained in a descriptive information 
system. Thus, you need some means of selecting the most appropriate 
indicators. Various criteria can be used to select the indicators most 
feasible and useful to maintain. Consider, for example, the following: 



• Availability : Is the informati^ already available and just^ needs 
to be pulled together, or must irlvKaata be colle<:ted? 

• Relevance: Of the different possible indicators of a concept of 
interest, which most closely reflects the concept? 

• Precision : How accurate is the indicator? 

• Cost: What is involved in collecting data on the indicator? Hov% 
much staff time? How much paperwork? *If it is a purchased test 
or survey instrument, how much do the copies cost, and how much 
does scoring cost? 

• Missing Data : How likely is it that you can obtain 'the information 
on al^ children or all families? Isliat percentage of the data is 

• likely toUfe missing? • 
^ % 

• Face Validity, : If audiences outside the school system --e .g . , funding 
agencies or the community--wi 1 1 review your data, which indicators 
will seem most valid to them? Which will seem less relevant? 

Figures 6 and 7 indicate methods for summarizing different criteria for 
indicators to show their relative advantages and disad\antaj;es '^x a elaiKC. 
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Indicator 



Frequency of 
Attendance at PAC 



Frequency of 
Attendance at 
Parent -Teacher 
Conferences 



Visits to School 



Volunteer 
'ome Tutoring 



Volunteer Teacher 
Aide Work 



Active 

^Participation 
at PAC 

1 



Characteristics of the Indicator 




Figure 6: lndica_tj)rs_of Parentjnv^l^^ 
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Indicator 



Standard Test 
Score 



Criterion- 
Referenced Test 
Score 



Characteristics- of the Indicator 



Availability 



1 



Teacher -Assigned 
Grades 



Teacher-Assigned 
Grades, JVriting 



Pages in Workbook 



Obi ect ives 
Accomplished 



Tardiness 



1 ruancy 



C.rade Retention 



^Referral to 
"^poc i.jl I ducnt ion 



?\<j) evance 
to 
General 
Concept 




Precision 



Estimated 
Cost of Data 
Collection 



Personnel 



Non- 
Personnel 



:enf c 



LiJ 

Percent of 
Mia-sing Data 



AA = Already available in someone's file; just needs to be pulled together 
NA = Not available but could be easily recorded; person and mechanism are in place 
53 Available with difficulty; new procedures wuld have to be established to get the data 
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Of course, that glance does not necessarily make selection easier, but it 
does provide the information needed for selection. 

STARTIN G SMALL - 

— { 

The exercises that we have described here should help identify the 
data that are most likely to be useful to you. However, they all rely on 
your guesses about ease of data collection, likelihood of use, and so on. 
And because they are based oh conjecture, it' would be wise not to initiate 
a large and complex longitudinal information system at once. Instead, begin 
with a small system, perhaps developing a file on one or two waves of child- 
ren, or even one or. two classrooms. After collecting data for a year or 
so^ you will no doubt want to modify your system. For example, the first 
time you create summary tables, you may wish you had gathered other 
information. In fact, if you show your tables to those who asked for this 
^information, they mafSell want still further infbrmation'to interpret the 
tables. The data on your first small group should then be revised as 
needed and maintained for at least two years. This mini-longitudinal system 
will enable you to discover other categories of children and measures of 
progress that you had not anticipated, which will lead to still further 
modifications in system design. 

SUMMARY 

This chapter has outlined exercises designed to help you determine your 

information needs: obtaining '^wish lists'' of information from those whom tiit 

longitudinal information system is designed to assist; refining those lists | 

by anticipating comparisons that might be made; and reducing the man\ kind? of 
f 

potentially useful information to thos^ that are affordable, practical, and 
most useful. Chapter 4 is devoted to characteristics of the information 
system itself that must also be determined in advance. 
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IV. CONSIDERATIONS IN DESIGNraG A DESCRIPTIVE INFORmTION SYSTEM 

Aside from diff icult^decisions about the information to be collected 

and maintained in a descriptive information system, the nature of the system 

itself must be decided. To some extent, system characteristics depend on 

f 

the information chosen; but since the system will cost mone^', the information 
you choose may also depend somewhat on cost limitations. This chapter 
therefore reviews the potential limitations of different information systems. 
An information system could be merely a file drawer of folders on indi- 
vidual children, or it could be a large computer with records stored on 
cards, tapes, or disks. Whether information is stored in file drawers or 
in computers, four characteristics define the system: si3e, organization, 
how it is gathered and entered, and how information is retrieved. Thes^ 
characteristics must be decided upon before the system is developed, and each 
depends partly on factors such as cost and feasibility, and partly, on who 

will use the information and how. 
SIZE 

The sire of fhe system means simply the number of pieces of informa- 
tion data points contained in the files. We saw earlier that a system 
describing children from kindergarten through sixth grade could contain 
rec9rds on 28 groups/grades of children. If each group contained 50 children, 
and one test score was tointained for each child at each grade, the system 
would contain 840 (30 x*28) test scores. Now suppose that each test has six 
subtelts, and that the system r-ccords those scores as well: it would then 
contain 840 x 6, or 5,040, test scores! 

The si:c of the system just described is defined by the numl-cr of chilJ- 
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^ ren (30 per year), the number of waves of children (seven), the fre'quency 
of»data collection (once a year for seven years), and the number of indica- 
tors (six subtest scores per chilT^per^Tear) . But this system already has 
a limitation. It does not retain data on children once they graduate from 
the sixth grade; that is, it never contains a full seven years of data for 

.more than one group of children. If all data on all classes were maintained, 

J 

the system would eventually contain 8,820 data points (30 children x 7 
classes x 7 testing times x 6 subtest scores) . 

This system is limited in other ways, for it contains no information 
on .children's educational experiences (which programs they were in, what 
textbooks they used, when or how often they received Title I services), on 
their personal characteristics (whether they are bilingual, their parents* 
income, their health), or on outcomes other than test scores (grade retention, 
tardiness or absence, social dev-^lopment) . Yet, as we have seen, such > 
information may be of interest for various comparisons that might be desired 
la-ter on. 

There are some mechanical ways of reducing the physical size of the files. 
For example, instead of retaining test booklets, scores could be transferred 
to individual summary sheets or note cards. This may seem a trivial modifi- 
cation, but 8,000 note cards take up^far less room than 8,000 test booklets. 
Or only class averages could be kept, so that 30 individual files are re- 
duced to one class average; but that may greatly reduce the potential for 
later comparison. The 30 children who were in the same early childhood 
program may have very different experiences later on, which- cannot be recorded 

without individual records. 
ORGANIZATION 

The way in which information is organized may also help save spnce, and 
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will certaxtrt5^ affect ease of access. For example, if a given record contains 
all the data on one child, longitudinal assessment of , individual children's 
progress will be easy, whereas comparisons of group. averages at any one time 
will require pulling those scores out of their longitudinal sequence and 
averaging them for each group of interest, ^ 

Textbooks on information systems usually use the**^rms subsystem and, 
integration in discussing organization. S ubsystems are certain sections of 
the files, and integration is the way in which they are linked, or cross- 
referenced. There are three ways of integrating subsystems of information: ^ 
horizontal integration, vertical integration, and longitudinal integration * 
(see Figure 8). Since each is suitable for a different Kind of analysis, 
it pays to think about what analyses you will want to do before establishing 
the system. 

% 

Horizonta 1 integration links subsystems that contain different kinds of 
info!nnation for example, one containing financial data with another con- 
taining achievement data. The best method of horizontal integration depends 
on how the data will be used. If, for example, all the questions asked will 
be about the relationship of these two domains, you may want to store the 
two sets of information together, say on a note card containing both cost 
and achievement data on a given child. But if important questions may relate 
only to cost (or only to achievement), that storage method will make data 
retrieval cumbersome; a better metk^NiJ^ould be to store the cost data 
separately and to code the achievement daH by the relevant cost categories, 
so that it can be connected as needed. 

Vertical integration is the summing up of data. If. for example, you 
want to average achievement across classrooms, school buildinr^s, or the entire 
district, data should be stored so as to facilitate aggrepatjon; in this 
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5rd GradeT 



VERTICAL 



2nd Grade 



1st Grade 



School A 

Rudyard Kipling 
Mary Shelley 



School B 

Susan Sontag 
Charles Dickens 




Eli 
James 



Age 



Brovming 



PPVT 



• File of Outcomes 



DDST 



HORIZONTAL 



2nd Grade 
"-^rvi ces 



1st Gra^e 
Services 




Kindergarten 
Services 

D I STAR 
3 Hours /Day 
Mrs, Jones 
Miss Caully 
Vision Exam 



File of Services 



Figure 8: Three Ways of Integrating Subsystems 
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case, each unit (classroom, school) may be a subsystem. Aggregation aci^ss 
units also means that each subsystem should flag the data that should not 
be aggregated. For example, if some children were pretested and then moved\^ 
to another school district, y«^ may want to exclude their scores when avera^ 
ging. Or if a flood caused one school building to close for a month, yoi 
may not want to average the scores of the children in that building with 
those of children in others. The individual records in each subsystem, then, 
should indicate any unusual circumstances that might affect the averages and 
render them uninte rpretabj^ tf*"'^'*'^ * . - jf, 

Longitudinal integration links information over time, for example, you 
may want to compare the achievement at third and at sixt.]vgrade of children who 
attended school in your district for their entire elementary school years. 
Since you do not want to compare all available scores, the grade-specific 
subsystems must be integrated in a way that allows you t^ identify the child- 
ren you are comparing. To preserve confidentiality, you might want to use 
some identification code rather than children's names^. 

The kinds of integration method used will depend on the content of the 

subsystem. For example, one subsystem might contain data on services, and 
another data on achievement; or one might contain first-grade and another 
Second-grade achievement data^ Subsystems are merely filed; but since a 
longitudinal data system may contain information on (a) different kinds of 
children ,^(b) different kinds or quantities of services, (c) different kinds 
of outcomes, and (d) different years, some thought should be given to how the 
subsystems should be defined, and how linked. 

One way to do this is to think about information that will not be 
summed or averaged. For example, if you think you will never (or rarely) 
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vant to pooljlata on English-speaking and Spanish-speaking children, then >ou 
may Waht to develop separate subsystems for these groups. Or if you will 
never want to pool data on children who did. and did not have early childhood 
programs, then again the two sets of data might be stored separately. If 
you know you will not average first-grade test scores with second-grade 
test scores, then these^ data should be separate subsystems, coded so that 
they can be "linked when Comparisons are desired. *^ 

Conversely, you might think about information that you will want to link. 
For example, if you know you will o'ften want to tie data on services to data 
on academic progress, you may want to store , both together in one subsystem. 
Test score averages can then be computed for each- kind of service delivered. 

_To define categories of service, you may need to develop some table out- 
lines for the kinds of comparison .lat might be made. Figures 9 and 10 
illustrate two such tables for defining subsystems. Each columnar row (or 
cell) can represent one subsystem in the total descriptive information system. 
HOW INFORMATION IS ENTERED INTO THE SYSTEM 

You have determined the size of your system and how it should be organi- 
zed; now you must consider how information will be collected and stored. 
It is one thing to decide to transfer^ scores onto note cards; quite another 
to find someone with the time to do it. If separate subsystems are to be^ 
developed for each building, perhaps someone in each building could record 
those data; this is espetially useful if the data are to be used mainly by 
the staffs in the different buildings. If, however, the data are intended 
for use by a central office, it might make more sense if someone at that 
office took charge of all records. In either case, there arc costs: in 
collecting the data, in transmitting them from the classroom to the person 
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(Combined- data for classes entering in 1974, 1975, 1976, and 1977) , 

Figure 9: Number of Children Receiving Different Amounts of Titl-e I Services 
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who maintains the' system, and in recordiag, storing, and updating them as 
needed. 

' HOW INFORMATION IS RETRIEVED 

The fourth dtjision about the design of your system is how the informa- 
tion^will be retrieved for use. For example, will you want regular reports 
on certain things, or' would you rather simply get information as you need 
it? Most people probably want both. For example, regular reports on 
^ enrollment and budgets may be useful for annual* budget-planning meetings, 
and regular reports of achievement may be useful for accounting to the 
public. On the other hand, many important decisions that cannot be 
anticipated may require special analyses. For these occasions, an ideal 
system would include someone who could respond to special requests' by pull- 
ing appropriate data, constructing whatever tables aje needed, and calculat- 
ing whatever averages or jDercents might be required. 

By this time it should be clear that not all spontaneous requests for 
data analysis will be equally easy to meet. The size and organization of 
the system will necessarily favor some kinds of analysis over others. The 
way the data are 'stored may make it extremely time-consuming and tedious to 
retrieve and organise information for uses that were not anticipated. 
^ PR-IVACY OF RECORDS 

' Longitijdinal information systems require that children can be identi- 
fied in some way so that their scores over time can be linked. In the past, 
when school records were considered the property of the schools, this was 
no problem. Today, however, children \s right to privacy is a controversial 
issue. .Schools can no longer completely control the disposition of school 
records; pareiits* and in some cases children, also have a say about the w^o 
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of grades, test scores, and teachers* comments written in students' files. 
The chief guarantee of parents* and children's rights in this matter is the 
Family Educational Rights and Privacy Act, often known as the Buci^ley Amend- 
ment, which became law in 1974. This Act, which applies to any institution 
receiving program funds through the Department of Education, has two main 
provisions: - ^ 

• To guarantee parents or guardians (and students over the age 
of 18 or who attend a post-sern^dary school) access to their 
own records 

• To restrict other people's access tcr-these records by re- 
quiring parental consent before the data can be released. 

The second provision is particularly significant for longitudinal informa- 
tion systems, which require that the identity of students be safeguarded. 
This pojes no problem in studies you conduct, since you'have a legitimate 
educatUital interest and are therefore not required to secure parental 
consent before reviewing a child's record. The use of records outside 
tne district, however, requires written permission--even if students' 
are replaced by numbers or other codes, since identities still could be 
traceable.' These rules should be taken into account in the design of the 
system, so that they can be observed while your own information needs are met 
COMPUTER VS> MANUAL SYSTEMS 

Wiile almost all other prices are rising, the price of computers is 
declining. And small computers that can perform a variety of tasks are 
becoming increasingly available. This trend suggests that it may be worth 
while fbr^you to take time to weigh the costs and benfits of a small com- 
puter relative to those of manuaj information systems. 

Computers -have many advantages. For example, if you knou that you will 
want certain tables produced each spring, you can program your computer to 
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produce them automatically; you can also program it to respond to nevs re- 
quests--9ay, the avefage reading scores of children who have been retained in 
second grade vs. those of children there for the first time. Computers can 
have built-in safeguards, so that only certain personnel have access to the 
information. And ©f course their calculations .are not subject to human error 

But computers have disadvantages, too. Though many of the nev% ones 
use languages very close to English, you will still need to train someone in 
using your new computer. And because th'ey are mechanical devices, they can 
break down, so some amount of repair and maintenance will be needed. 

Finding out about computers is as simple as looking in the yellow pages 
under Data Processing Systems . Your investigation should include considera- 
tion of the following. 

• Capability to Meet Your Needs : You should be fairly sure of 
your system needs (si^ze, organization, etc.) so that you can 
assess the specific capabilities of different computers to 
meet those needs. 

• , Small Computer vs. Terminal : If you get a small computer, 
you have the whole thing right in your building. ^If you 
get a terminal, you have telephone access to a" much larger 
computer that resides elsewhere. Usually you buy computers, 
but you rent terminals. Terminals vary in the extent to 
which the files of information can be stored in the termi- 
nal itself as opposed to in the main computer. 

• Maintenance : You should seek a system that includes a main- 
tenance service agreement as part of the purchase or lease. 

» - Training : Check to see how complicated the computer is to 
program and use, and how much staff training will be needed. 
Also check to see how much training will be offered as part 
of the sale or lease. 

• Characteristics of the Files : The information you keep on 
file can be stored in the machrhe iself or on ouxiliar> tHjuip- 

^^^t such as cards, tapes, or '^floppy disks.*' It is not safe 

keep it only in the machine, for a power failure could 
Wean the loss of the (yitire file. Most people today prefer 
floppy disks to other storage methods. Be <^ure to invo?^t igat f 
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the advantages and disadvantages of each for (a) entering 
the information, (b) revising or updating it, and (c) re- 
trieving it. 

• Technical Assistance : We suggested earlier that you should 
start small,, a^^J i..-.. the size and scope of your system 
only after you are sure of your needs. Check which computer 
companies offer assistance in determining your needs, or pro- 
gramming the computer to meet them. Don't buy until you are 
sure that you kfiow what you want and that this system is best 
suited to the uses you have defined. 

WERE TO GO FROM HERE 

We have described several characteristics of information systems 
that must be determined before the system is designed: size, organization, 
hou inform^ion is entered into the system, how it is retrieved from the 
system, privacy of records, and computer vs. manual systems. These 
characteristics are interrelated. System size and organization will depend 
on the sraff or resources available, on the form in which information will 
be retrieved, and on the intended use. Since' few school districts can 
afford a system containing ail possible useful information, system charac- 
teristics are partly a function of how large and sophisticated a ^system the 
district can afford. Thus, the costs and benefits of different system designs 
must be compared, relative to the value of the information they will provide ^ 
for you. ^ 

What do you do once you have read this book"" Start with a lot of thin; 
ing. Make some of the wish lists sugg>?^ed here to help you estimate, your 
needs. Talk to people. Consiilt with your Title I Technical Assistance Ccnti.^ 
staff. Then start a small trial information system. See how you use it. 
See what quest ions'peopl e a^k of it. Gradually, you will refine and re\i^c 
your system as you enlarge it, so that eventual ly--perhaps as muJi as-fjw 
years from now--you will have a useful information system. - ) 
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NOTES ON SOURCES OF FURTH^ INFORf-lATION 



Writing Computer Progra ms in BASIC . V/ellesley Hills, MA: Applied Decision 
Systems, 1970. 



This book is a manual on how to use a particular computer proj^ram 
system. The BASIC program is designed especially for use on computer 
terminals, and th^ manual provides guidance on how to use a terminal, 
and how to link it with a computer, as well as on how to use the BASIC 
sys{eT^^s on your terminal. The total book is just over 50 pages. 



Baltes, Paul B., Reese, H.W., and Nesselroade, J.R. Life- Span Developmental 
Psychology: Introduction to Rese arch Methods. Nbnterey, CA: Brooks/ 
Cole, 1977. \ 

This convenient little paperback book provide? a readable introduction 
to the longitudinal concepts and the various methodological issues involved 
in studying and analyzing developmental change. Not only do these authors 
discuss research design and control issues, but they also have chapters on 
measurement problems in longitudinal data-collection efforts and data analy- 
sis and interpretation. The book is divided into five major parts, and 
one of them deals exclusively with descriptive systems. 

Campbell, Donald. T. Temporal changes in treatment-effect correlations: A 
quarsi-experimental model for institutional records and longitudinal 
studies. Published in the Proceedings of the 1970 Invitational Confer- 
ance on Testing Problems . Princeton, NJ: Educational Testing Service. 
1970 (pp. 93-110). 

» 

This 30-page article discusjses the problems of determining the effects 
of special programs when the ideal conditions of an ''experiment'' cannot bo 
realized. Campbell demonstrates the use of correlation coef f icicnt s' as tool 
to assist the analysis of data, and provides some very well thou^[ht-out 
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discussions about how to interpret different longitudinal patterns 
in the data. ^ * 

Diederich, Paul'B, Pitfalls in the measurement of gains in- achievement . 
School Review , 1956 (Vol, 64), pp. 59-63, 

Although several articles have. been written on this topic since 
Diederich's, this short and straightforward article bffers a good statement 
of some of the most prevalent problems in interpreting data on children's 
gains. ^ . 

Gro<;sman, Alvin, and Howe, R.L. Data Processing in Education . Chicago: 
Educational Methods, Inc., 196r! 

Although much of the information about particular computers is now out 

of date, this book offers a variety of helpful advice on how to make the 

initial decisions, such as selecting a computer, estimating what it will be 

able to do -for you. and managing the transition from a manual filing sys- 

tern to an automated system. The book also discusses some of the economic 

and personnel considerations in using computers. Most of these topics are 

discussed in the first half of the book. The second half describes various 

applications of automated record-keeping in education, such as test" scoring 

and reporting, attendance accounting and bookkeeping. 

t 

\ 

Nie, Norman H., Hull, C. Hadlai, Jenkins, J.G., Steinbrenner, K., and Bent, 
Dale H. SPSS: Statistical Package for the Social Scien ces. New York: 
McGraw-Hill, 1975. ' ~ 

This book is actually a manual on how to use a particular package of ' 
computer programs, the SPSS. The SPSS computer programs are most widely 
used by researchers, since they include a wide variety of complex statis- 
tical analyses. The bulk of this very large book describes the specific 
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procedures for using these programs to conduct specific analyses, but the 



information about coding data so that these SPSS programs can be used to 
analyze it. These first two chapters are particularly worth reading if you 
think you may want to conduct very complex analyses and may therefore want 
to use the SPSS programs on your computer. 

Tuckman, Bruce W. Conducting Educational Research . New York* Harcourt 
Brace Jovanovich, 1972. 



This textbook is an excellent introduction to a variety of different 
aspects of educational research. Three chapters may be particularly use- 
ful to those who are planning to develop longitudinal information systems: 
Chapters 5 and 6 describe some of the considerations involved in making 
comparisons among groups, and Chapter 10 gives very explicit guidance on 
how to code data for computer data processing. It also shows examples of 
hand-written siommaries of data, key-punched cards, and printouts. (Chapter 
10 is about 30 pages.) 



first two chapters (a total of 27 pages out of over 600) describe general 
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